The human visual system perceives the same range of colors under a wide variety of scene illuminations. A white piece of paper remains resolutely white independent of the color of the illuminant (the color of the light under which the piece of paper is viewed). In contrast, color imaging systems (e.g., digital cameras) are less color constant in that they will often incorrectly infer the color of the illuminant. Unless the influence of the color of the illuminant is compensated, the digital camera cannot acceptably reproduce actual scene colors. The process of “correcting” the image data to compensate the effect of the illuminant color is commonly referred to as white balancing.
In white balancing, the color of the scene illumination either is measured or is estimated from the image data, and then the image data is adjusted to compensate for the effect of the illuminant. Because it is not practical to equip each camera with a dedicated illumination sensor and then expect users to calibrate to a white reference every time a picture is taken, conventional cameras typically estimate the illuminant color from the image data.
According to one conventional white balancing method, all of the pixel color values in an image are averaged, and the image data is then adjusted so that the average of the pixel color values is gray. According to another conventional white balancing method, the brightest spot (a specularity) in an image is presumed to be the light source, the color of that specularity is presumed to be the color of the illuminant, and the image data is adjusted to compensate for the color of the illuminant. These types of methods are simple to implement but unfortunately they fail in many scenarios because they are based on assumptions that do not always hold true. For example, the average color of a scene may not actually be gray, and a specularity may not appear in an image (and when present may be difficult to find).
Other attempts to address the illuminant estimation problem use a probabilistic framework. Given an input image Cim, the objective of a probabilistic approach is to recover Pr(E|Cim)—the probability that E was the scene illuminant given Cim. The illuminant hypothesis Ei that produces the largest Pr(E|Cim) is the estimated scene illuminant. According to Bayes' Rule:
                                          Pr            ⁡                          (                              E                ❘                                  C                  im                                            )                                =                                                    Pr                ⁡                                  (                                                            C                      im                                        ❘                    E                                    )                                            ⁢                              Pr                ⁡                                  (                  E                  )                                                                    Pr              ⁡                              (                                  C                  im                                )                                                    ;                            (        1        )            where Pr(E) is the probability that the scene illuminant is E, and Pr(Cim|E) is the probability of observing the image Cim under illuminant E.
Pr(Cim) is a constant regardless of the illuminant, so it can be omitted from equation (1) without changing the outcome of the maximum probability estimation:Pr(E|Cim)=Pr(Cim|E)Pr(E);  (2)where Pr(E) is the probability of illuminant E. In most cases, Pr(E) is assigned an equal value for all illuminant colors. The likelihood function Pr(Cim|E) is typically estimated by assuming that the chromaticity values of the pixels in an image are independent of each other, so that Pr(Cim|E) can be replaced in equation (2) by the product of the conditional probabilities of the pixel given illuminant E:
                                          Pr            ⁢                          (                              E                ❘                                  C                  im                                            )                                =                                    [                                                ∏                                                            ∀                      c                                        ⁢                                          ∈                                              C                        im                                                                                                                                                    ⁢                                                                  ⁢                                  Pr                  ⁡                                      (                                          c                      ❘                      E                                        )                                                              ]                        ⁢                          Pr              ⁡                              (                E                )                                                    ;                            (        3        )            where c is the color (chromaticity) of a pixel in image Cim, and Pr(c|E) indicates how “probable” it is that the camera would capture a color c under the illuminant color E (that is, given an illuminant color E, what is the probability that color c would be observed in the image).
In order to generate Pr(c|E), a correlation matrix is created to correlate possible image colors with potential colors of scene illuminants. The correlation matrix characterizes, for each illuminant color considered, the range of possible image colors that can be observed under that illuminant color. In other words, a “camera gamut” for each potential illuminant color is constructed.
To implement a correlation matrix in a camera, training image data is used to create a two-dimensional (2D) or even three-dimensional (3D) color histogram for each illuminant color considered, and the histograms are installed in the camera's memory as lookup tables. If pixel color is represented using eight (8) bits per pixel, and the probabilistic score Pr(c|E) is represented in an 8-bit format, a 2D lookup table requires 64,000 bytes of memory per illuminant color, and a 3D lookup table requires 16,000,000 bytes of memory per illuminant color. Because multiple illuminant colors are typically considered, an extensive amount of memory can be consumed by a conventional correlation matrix.